Creating virtual machine snapshots without interfering with active user sessions

ABSTRACT

Systems and methods for creating virtual machine snapshots. An example method comprises: receiving a request to create a snapshot of a virtual machine running on a host computer system; protecting from modification a plurality of virtual memory pages of the virtual machine; responsive to detecting an attempt to modify a virtual memory page of the plurality of memory pages, copying the virtual memory page to a queue residing in a random access memory (RAM) of the host computer system; making the virtual memory page writable; retrieving the virtual memory page from the queue; writing the virtual memory page to a disk of the host computer system; and responsive to exhausting the queue, completing creation of the snapshot of the virtual machine.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/454,090 filed on Mar. 9, 2017, entitled “Creating Virtual MachineSnapshots without Interfering with Active User Sessions,” the entirecontent of which is incorporated by reference herein.

TECHNICAL FIELD

The present disclosure is generally related to computer systems, and isspecifically related to systems and methods for creating snapshots ofvirtual machines.

BACKGROUND

Virtualization may be viewed as abstraction of hardware components intological objects in order to allow a computer system to execute varioussoftware modules, for example, multiple operating systems, concurrentlyand in isolation from other software modules. Virtualization may beachieved by running a software layer, often referred to as a “virtualmachine monitor,” above the hardware and below the virtual machines. Thevirtual machine monitor may abstract the physical layer and present thisabstraction to virtual machines to use, by providing interfaces betweenthe underlying hardware and virtual devices of virtual machines. Forexample, processor virtualization may be implemented by the virtualmachine manager scheduling time slots on one or more physical processorsfor a virtual machine, rather than a virtual machine actually having adedicated physical processor.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of examples, and not by wayof limitation, and may be more fully understood with references to thefollowing detailed description when considered in connection with thefigures, in which:

FIG. 1 depicts a high-level diagram of an example computer system 100 inwhich the example methods of creating virtual machine snapshots may beimplemented, in accordance with one or more aspects of the presentdisclosure;

FIG. 2 schematically illustrates an example sequence of operations thatmay be performed by the virtual machine snapshot creation module 160 forcreating a virtual machine snapshot, in accordance with one or moreaspects of the present disclosure;

FIGS. 3-5 depict flow diagrams of example methods of creating virtualmachine snapshots, in accordance with one or more aspects of the presentdisclosure; and

FIG. 6 depicts a block diagram of an example computer system operatingin accordance with one or more aspects of the present disclosure.

DETAILED DESCRIPTION

Described herein are systems and methods of creating virtual machinesnapshots without interfering with active user sessions.

A virtual execution environment implemented by a host computer systemmay comprise a virtual machine monitor (VMM) facilitating execution ofone or more virtual machines, each of which may run a guest OS managingone or more applications. The VMM may emulate the underlying hardwareplatform (e.g., the x86 platform), including emulating the processor,memory, and peripheral devices (such as network interface controllers,hard disk controllers, etc.). In various illustrative examples, thevirtual execution environment may be employed for executing code thathas originally been developed for platforms different from the hostplatform.

In certain implementations, the virtual execution environment may beimplemented using hardware-assisted virtualization features of the hostplatform. Such hardware-assisted virtualization features may enableexecuting, at an elevated privilege level, a VMM that acts as a host andhas the full control of the processor and other platform hardware. TheVMM presents a virtual machine with an abstraction of one or morevirtual processors. The virtual machine implements a softwareenvironment which may be represented by a stack including a guestoperating system (OS) and application software. Each virtual machine mayoperate independently of other virtual machines and use theVMM-facilitated interface to the processors, memory, storage, graphics,and I/O provided by the host platform, while the VMM may retainselective control of processor resources, physical memory, interruptmanagement, and input/output (I/O).

“Virtual machine snapshot” herein shall refer to a non-volatile memorycopy of the virtual machine state, which may include virtual processors,the virtual random access memory (RAM), disks associated with thevirtual machine, virtual peripheral devices, and VMM internal structuresrequired for guest operating system virtualization. The non-volatilememory for saving the snapshot may be provided by a disk of the hostcomputer system, although other implementations of the non-volatilememory also fall within the scope of the disclosure.

Conventional methods of creating virtual machine snapshots may requireinterrupting any currently active user sessions and/or de-initializingthe virtualization services in order to save the content of the virtualRAM to the disk. Such interruption and de-initialization may beundesirable, especially in situations where the virtual machine supportsan interactive environment, as they would adversely affect the end userexperience and the overall system efficiency. The systems and methods ofthe present disclosure alleviate these and other deficiencies ofconventional snapshot creation methods, by utilizing a host RAM buggerfor queuing the virtual memory pages, and then asynchronously saving thequeued memory pages to the disk. Since the queueing operation onlyinvolves memory copying, it does not introduce any perceivableinterruptions to the currently active user sessions. In order to furtherreduce the amount of disk I/O operations, the memory pages may becompressed before being saved to the disk.

Responsive to receiving a request to create a snapshot of a runningvirtual machine, the snapshot creation logic may write-protect thevirtual machine memory pages, e.g., by clearing the correspondingwritable flags in the paging table. The snapshot creation logic may thenstart queuing the virtual memory pages to a queue residing in a host RAMbuffer. Responsive to successfully queueing a memory page, the page maybe made writable, e.g., by setting the corresponding writable flag inthe paging table. Since the queueing operation does not interfere withany currently active user sessions, this phase is referred to assynchronous (with respect to the user session).

A virtual machine's attempt to modify a write-protected virtual memorypage would trigger a virtual machine (VM) exit, thus yielding theexecution flow control to the VMM, which in response may append thecontent of the virtual memory page to the queue, make the page writable,and resume the virtual machine execution without introducing perceivabledelays into any currently active user sessions. The queued virtualmemory pages may then be asynchronously retrieved from the host RAMbuffer and saved to the disk, thus freeing up the space in the hostmemory buffer for subsequent queueing of eventually modified virtualmachine memory pages.

In certain implementations, the virtual memory pages may be compressedbefore being saved to the disk. Compressing the memory pages may reducethe amount of disk I/O operations at the cost of increasing theprocessor utilization. The snapshot creation logic may determine whetherone or more queued memory pages should be compressed before being savedto the disk, in order to minimize the overall snapshot creation timeunder the current processor load.

In an illustrative example, the snapshot creation logic may employtwo-stage queueing of virtual memory pages: one host RAM buffer may beutilized for saving the original memory pages, which are then compressedand saved into another host RAM buffer, for being finally saved to thedisk. The compression and disk saving operations may be performedasynchronously with respect to the initial queueing of the virtualmemory pages. Responsive to exhausting the queue(s), the snapshotcreation process may be completed.

Thus, the systems and methods described herein represent improvements tothe functionality of general purpose or specialized computing devices,by utilizing the host RAM for queuing the virtual memory pages that havebeen modified since the last snapshot creation, and then asynchronouslysaving the queued memory pages to the disk, in order to create a virtualmachine snapshot, as described in more detail herein below. The systemsand methods described herein may be implemented by hardware (e.g.,general purpose and/or specialized processing devices, and/or otherdevices and associated circuitry), software (e.g., instructionsexecutable by a processing device), or a combination thereof. While theexamples presented herein describe virtual machines operating as memoryaccessing agents, non-virtualized memory agents also fall within thescope of this disclosure. Various aspects of the above referencedmethods and systems are described in details herein below by way ofexamples, rather than by way of limitation.

FIG. 1 depicts a high-level diagram of an example computer system 100 inwhich the methods of virtual machine snapshot creation may beimplemented in accordance with one or more aspects of the presentdisclosure. The computer system 100 may include one or more centralprocessing units (CPU) 110, also referred to as “processors” herein,which may be communicatively coupled to one or more memory devices 115and one or more input/output (I/O) devices 120 via a system bus 125. Inan illustrative example, an I/O device 120 may represent a diskcontroller that facilitated communications of CPU 110 and othercomponents with one or more disks 130.

“Processor” herein refers to a device capable of executing instructionshandling data in registers or memory, encoding arithmetic, logical,control flow, floating point or I/O operations. In one illustrativeexample, a processor may follow Von Neumann architectural model and mayinclude an arithmetic logic unit (ALU), a control unit, and a pluralityof registers. In a further aspect, a processor may be a single coreprocessor which is typically capable of executing one instruction at atime (or process a single pipeline of instructions), or a multi-coreprocessor which may simultaneously execute multiple instructions. Inanother aspect, a processor may be implemented as a single integratedcircuit, two or more integrated circuits, or may be a component of amulti-chip module. A processor may also be referred to as a centralprocessing unit (CPU). “Memory device” herein refers to a volatile ornon-volatile memory, such as RAM, ROM, EEPROM, or any other devicecapable of storing data. “I/O device” herein refers to a device capableof inputting and/or outputting binary data. In an illustrative example,an I/O device may be provided by a network interface controller (NIC) ora block I/O device, such as a hard disk controller.

In accordance with one or more aspects of the present disclosure, thecomputer system 100 may implement a virtual execution environment thatis employed, e.g., for executing the code that has originally beendeveloped for a platform which is different from the host platform. Thevirtual execution environment may comprise one or more virtual machines140A-140N, each of which may run a guest OS managing one or moreapplications. Lifecycles of the virtual machines 140A-140N may bemanaged by the VMM 150. In certain implementations, the host computersystem 100 may implement a virtual machine snapshot creation module 160that may be employed to create virtual machine snapshots withoutinterfering with active user sessions. In various illustrative examples,the virtual machine snapshot creation module 160 may be implemented as auserspace module, a kernel module, a combination thereof, or an embeddedpart of a complex virtualization module.

FIG. 2 schematically illustrates an example sequence of operations thatmay be performed by the virtual machine snapshot creation module 160 forcreating a virtual machine snapshot. As schematically illustrated byFIG. 2, the virtual machine state may comprise a plurality of virtualmemory pages 200. For at least some of the virtual memory pages 200,FIG. 2 schematically illustrates the sequence of states through whichthe virtual memory page is transitioning: utilized by the guest OS 202,protected from modification 204, requested to be modified by guest OS206, queued for compression 208, compressed and queued for saving todisk 210, and saved to disk 212.

At the time 215, a new snapshot may be initiated (e.g., by a snapshotcreation request). Any active user sessions may be temporarily suspended(e.g., by pausing the host processing threads emulating the virtualmachine processors). While virtual processors in the suspended statecease executing guest instructions, the snapshot creation module 160 maywrite-protect the virtual machine memory pages 200, e.g., by clearingthe corresponding writable flags in the paging table (e.g. bit R/W ofpage table entry in Intel x86 architecture) and/or by disabling writesto particular page using platform-specific hardware-assistedvirtualization structures (like the nested page table structure of IntelEPT extension). The snapshot creation module 160 may then start queueingthe virtual memory pages 220A-220N for compression. The compressionqueue may reside in one or more host RAM buffers (not shown in FIG. 2for clarity and conciseness). Responsive to successfully saving to adisk, successfully compressing or successfully queueing one or morememory pages, the pages may be made writable, e.g., by setting thecorresponding writable flags in the paging table and/or usingplatform-specific hardware-assisted virtualization features.

At the time 225, the previously suspended user sessions may be resumed(e.g., by scheduling the host processing threads emulating the virtualmachine processors). Eventually, the guest OS may attempt to modify oneor more write-protected memory pages 230A-230K. A virtual machine'sattempt to modify a write-protected virtual memory page would trigger avirtual machine (VM) exit, thus yielding the execution flow control tothe VMM, which in response may cause the snapshot creation module 160 toqueue the virtual memory page 240A-240K for compression, make the pagewritable, and resume the virtual machine execution without introducingperceivable delays into any currently active user sessions.

Virtual memory pages that have been queued for compression 240A-240K maythen be asynchronously, with respect to the queueing operation,retrieved from the compression queue, compressed, and placed intoanother queue for being saved to the disk (250A-250K), thus freeing upthe space in the compression queue.

Virtual memory pages that have been queued for saving to disk 250A-250Kmay then be asynchronously, with respect to the queueing operation,retrieved from the disk queue, and saved to the disk (260A-260M), thusfreeing up the space in the disk queue. In order to increase theoperational efficiency, saving the virtual memory pages to the disk maybe performed by a system call bypassing an input/output (I/O) cache ofthe host computer system. Responsive to exhausting both compression anddisk queues, the snapshot creation may be completed (270).

As the term “queue” suggests, the memory pages may be retrieved from thequeue in the same order in which they were initially queued, thusimplementing the first in-first out (FIFO) processing strategy. Incertain implementations, retrieval of the memory pages from each queuemay be performed by a dedicated processing thread. In an illustrativeexample, queueing the pages for compression may be performed by a firstprocessing thread, retrieving the pages from the compression queue maybe performed by a second processing thread, and saving the pages to thedisk may be performed by a third processing thread, so that each ofthose operations is performed asynchronously with respect to the otheroperations.

In certain implementations, the compressing thread (if compression isenabled) or the disk writing thread (if compression is disabled) maytraverse the guest memory and sequentially process the guest memorypages. If the page that is currently being processed is found in thecompression queue (or the disk queue if the compression is disabled),then the queued page may be used for compression or saving to disk.Alternatively, if the page is not found in the compression queue (ordisk queue), the page content may be retrieved from the guest memory,compressed, and saved to disk.

In certain implementations, the snapshot creation module 160 may employa single-stage queuing of virtual memory pages, such that all virtualmemory pages are directly, without being compressed, queued to the diskqueue. Alternatively, the compression operation may selectively beperformed for certain memory pages based on comparing the amount ofavailable host RAM for queueing the memory pages and the currentprocessor utilization, in order to minimize the overall snapshotcreation time under the current processor load. In an illustrativeexample, if the amount of available host RAM for queueing the memorypages falls below a low water threshold, the virtual memory pages shouldbe queued for compression. In another illustrative example, if theamount of available host RAM for queueing the memory pages exceeds ahigh water threshold, the virtual memory pages may be saved to the diskin the uncompressed state. In yet another illustrative example, if theamount of available host RAM for queueing the memory pages exceeds thelow water threshold but falls below the high water threshold, thevirtual memory pages may be compressed if the current processorutilization falls below certain threshold.

Thus, responsive to detecting a modification attempt with respect to oneor more virtual memory pages, the snapshot creation module 160 maydetermine whether to apply the compression operation to one or morememory pages based on evaluating a function of the available host RAMand the current processor utilization. Depending upon the value of thefunction meeting a falling below of certain threshold values, thesnapshot creation module 160 may copy the virtual memory pages to thecompression queue or to the disk queue.

Example methods that may be performed by the snapshot creation module160 for creating virtual machine snapshots are described herein belowwith references to FIGS. 3-5.

FIG. 3 depicts a flow diagram of an example method 300 of creatingvirtual machine snapshots, in accordance with one or more aspects of thepresent disclosure. The method 300 and/or each of its individualfunctions, routines, subroutines, or operations may be performed by oneor more processing devices of the computer system (e.g., computer system1000 of FIG. 6) implementing the method. The method 300 may be performedby two or more processing threads, each thread executing one or moreindividual functions, routines, subroutines, or operations of themethod. In the illustrative example of FIG. 3, the first processingthread 302 performs memory page queueing operations, while the secondprocessing thread 304 retrieves the pages from the queue. In certainimplementations, the processing threads implementing the method 300 maybe synchronized (e.g., using semaphores, critical sections, and/or otherthread synchronization mechanisms).

The first processing thread may, at block 310, receive a request tocreate a snapshot of a virtual machine running on the host computersystem, as described in more detail herein above.

At block 320, the first processing thread may protect from modificationa plurality of virtual memory pages of the virtual machine, e.g., byclearing the corresponding writable flags in the paging table and/orusing platform-specific hardware-assisted virtualization features, asdescribed in more detail herein above.

Responsive to detecting, at block 330, an attempt to modify a virtualmemory page of the plurality of memory pages, the first processingthread may, at block 340, copy the virtual memory page to a queueresiding in one or more RAM buffers of the host computer system, asdescribed in more detail herein above.

At block 350, the first processing thread may make the virtual memorypage writable, e.g., by setting the corresponding writable flags in thepaging table and/or using platform-specific hardware-assistedvirtualization features, as described in more detail herein above.

The second processing thread may, asynchronously with respect to thequeueing operations performed by the first processing thread, at block360, retrieve the virtual memory page from the queue, as described inmore detail herein above.

At block 370, the second processing thread may save the virtual memorypage to a disk of the host computer system. Since the virtual memorypage has been made writable immediately after having been queued, thedisk saving operation does not involve interrupting the user sessionassociated with the virtual machine, as described in more detail hereinabove.

Responsive to determining, at block 380, that the queue contains nopages, the second processing thread may, at block 390, complete thesnapshot creation, and the method may terminate.

FIG. 4 depicts a flow diagram of another example method 400 of creatingvirtual machine snapshots, in accordance with one or more aspects of thepresent disclosure. The method 400 and/or each of its individualfunctions, routines, subroutines, or operations may be performed by oneor more processing devices of the computer system (e.g., computer system1000 of FIG. 6) implementing the method. The method 400 may be performedby two or more processing threads, each thread executing one or moreindividual functions, routines, subroutines, or operations of themethod. In the illustrative example of FIG. 4, the first processingthread 402 performs memory page queueing operations, while the secondprocessing thread 404 retrieves the pages from the compression queue andthe third processing thread 406 retrieves the pages from the disk queue.In certain implementations, the processing threads implementing themethod 400 may be synchronized (e.g., using semaphores, criticalsections, and/or other thread synchronization mechanisms).

The first processing thread may, at block 410, receive a request tocreate a snapshot of a virtual machine running on the host computersystem, as described in more detail herein above.

At block 415, the first processing thread may protect from modificationa plurality of virtual memory pages of the virtual machine, e.g., byclearing the corresponding writable flags in the paging table and/orusing platform-specific hardware-assisted virtualization features, asdescribed in more detail herein above.

Responsive to detecting, at block 420, an attempt to modify a virtualmemory page of the plurality of memory pages, the first processingthread may, at block 425, copy the virtual memory page to a first queuefor compression. The first queue may reside in one or more RAM buffersof the host computer system, as described in more detail herein above.

At block 430, the first processing thread may make the virtual memorypage writable, e.g., by setting the corresponding writable flags in thepaging table and/or using platform-specific hardware-assistedvirtualization features, as described in more detail herein above.

The second processing thread, asynchronously with respect to the firstqueueing operation, may, at block 440, retrieve the virtual memory pagefrom the first queue, as described in more detail herein above.

At block 445, the second processing thread may compress the virtualmemory page and, at block 450, place the compressed virtual memory pageinto a second queue for being saved to disk. The second queue may residein one or more RAM buffers of the host computer system, as described inmore detail herein above.

The third processing thread may, asynchronously with respect to thefirst and second queueing operations, at block 460, retrieve thecompressed virtual memory page from the second queue, as described inmore detail herein above.

At block 465, the third processing thread may save the compressedvirtual memory page to a disk of the host computer system. Since thevirtual memory page has been made writable immediately after having beenqueued, the disk saving operation does not involve interrupting the usersession associated with the virtual machine, as described in more detailherein above.

Responsive to determining, at block 470, that both queues contain nopages, the third processing thread may, at block 475, complete thesnapshot creation, and the method may terminate.

FIG. 5 depicts a flow diagram of another example method 500 of creatingvirtual machine snapshots, in accordance with one or more aspects of thepresent disclosure. The method 500 and/or each of its individualfunctions, routines, subroutines, or operations may be performed by oneor more processing devices of the computer system (e.g., computer system1000 of FIG. 6) implementing the method. The method 500 may be performedby two or more processing threads, each thread executing one or moreindividual functions, routines, subroutines, or operations of themethod. In the illustrative example of FIG. 5, the first processingthread 502 performs memory page queueing operations, while the secondprocessing thread 504 sequentially traverses the guest memory pages andretrieves the modified pages from the queue. In certain implementations,the processing threads implementing the method 500 may be synchronized(e.g., using semaphores, critical sections, and/or other threadsynchronization mechanisms).

The first processing thread may, at block 510, receive a request tocreate a snapshot of a virtual machine running on the host computersystem, as described in more detail herein above.

At block 515, the first processing thread may protect from modificationa plurality of virtual memory pages of the virtual machine, e.g., byclearing the corresponding writable flags in the paging table and/orusing platform-specific hardware-assisted virtualization features, asdescribed in more detail herein above.

Responsive to detecting, at block 520, an attempt to modify a virtualmemory page of the plurality of memory pages, the first processingthread may, at block 525, copy the virtual memory page to a queueresiding in one or more RAM buffers of the host computer system, asdescribed in more detail herein above.

At block 530, the first processing thread may make the virtual memorypage writable, e.g., by setting the corresponding writable flags in thepaging table and/or using platform-specific hardware-assistedvirtualization features, as described in more detail herein above.

The second processing thread may, at block 540, asynchronously withrespect to the queueing operations performed by the first processingthread, initialize the pointer of the guest memory pages to point to thestart of the guest physical address space. In an illustrative example,the second processing thread may traverse the guest memory pages in theorder of their respective guest physical addresses.

Responsive to determining, at block 545, that the guest memory pageidentified by the current value of the pointer is found in the queue,the second processing thread may, at block 550, retrieve the content ofthe memory page from the queue; otherwise, at block 555, the secondprocessing thread may retrieve the content of the memory page from theguest memory.

At block 560, the second processing thread may save the virtual memorypage to a disk of the host computer system. In certain implementations,the disk saving operation may be preceded by compressing the content ofthe memory page, as described in more details herein above. Since thevirtual memory page has been made writable immediately after having beenqueued, the disk saving operation does not involve interrupting the usersession associated with the virtual machine, as described in more detailherein above.

At block 565, the second processing thread may increment the pointeremployed to traverse the guest memory pages. In an illustrative example,the value of the pointer may be increased by the size of the guestmemory page.

Responsive to determining, at block 570, that the guest address spacehas not yet been exhausted (e.g., that the value of the pointer fallsshort of the maximum value of the guest physical address), the methodmay loop back to block 545; otherwise, at block 575, the secondprocessing thread may complete the snapshot creation, and the method mayterminate.

FIG. 6 schematically illustrates a component diagram of an examplecomputer system 1000 which may perform any one or more of the methodsdescribed herein. In various illustrative examples, the computer system1000 may represent the example computer system 100 of FIG. 1.

The example computer system 1000 may be connected to other computersystems in a LAN, an intranet, an extranet, and/or the Internet. Thecomputer system 1000 may operate in the capacity of a server in aclient-server network environment. The computer system 1000 may be apersonal computer (PC), a set-top box (STB), a server, a network router,switch or bridge, or any device capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that device. Further, while only a single example computer system isillustrated, the term “computer” shall also be taken to include anycollection of computers that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of the methodsdiscussed herein.

The computer system 1000 may comprise a processing device 1002 (alsoreferred to as a processor or CPU), a main memory 1004 (e.g., read-onlymemory (ROM), flash memory, dynamic random access memory (DRAM) such assynchronous DRAM (SDRAM), etc.), a static memory 1006 (e.g., flashmemory, static random access memory (SRAM), etc.), and a secondarymemory (e.g., a data storage device 1018), which may communicate witheach other via a bus 1030.

The processing device 1002 represents one or more general-purposeprocessing devices such as a microprocessor, central processing unit, orthe like. More particularly, the processing device 1002 may be a complexinstruction set computing (CISC) microprocessor, reduced instruction setcomputing (RISC) microprocessor, very long instruction word (VLIW)microprocessor, processor implementing other instruction sets, orprocessors implementing a combination of instruction sets. Theprocessing device 1002 may also be one or more special-purposeprocessing devices such as an application specific integrated circuit(ASIC), a field programmable gate array (FPGA), a digital signalprocessor (DSP), network processor, or the like. In accordance with oneor more aspects of the present disclosure, the processing device 1002may be configured to execute instructions implementing the methods300-500 of processing virtual machine I/O requests by virtualizationextension modules.

The computer system 1000 may further comprise a network interface device1008, which may be communicatively coupled to a network 1020. Thecomputer system 1000 may further comprise a video display 1010 (e.g., aliquid crystal display (LCD), a touch screen, or a cathode ray tube(CRT)), an alphanumeric input device 1012 (e.g., a keyboard), a cursorcontrol device 1014 (e.g., a mouse), and an acoustic signal generationdevice 1016 (e.g., a speaker).

The data storage device 1018 may include a computer-readable storagemedium (or more specifically a non-transitory computer-readable storagemedium) 1028 on which is stored one or more sets of executableinstructions 1026. In accordance with one or more aspects of the presentdisclosure, the executable instructions 1026 may comprise executableinstructions encoding various functions of the methods 300-500 ofcreating virtual machine snapshots.

The executable instructions 1026 may also reside, completely or at leastpartially, within the main memory 1004 and/or within the processingdevice 1002 during execution thereof by the computer system 1000, themain memory 1004 and the processing device 1002 also constitutingcomputer-readable storage media. The executable instructions 1026 mayfurther be transmitted or received over a network via the networkinterface device 1008.

While the computer-readable storage medium 1028 is shown in FIG. 6 as asingle medium, the term “computer-readable storage medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database, and/or associated caches and servers) thatstore the one or more sets of VM operating instructions. The term“computer-readable storage medium” shall also be taken to include anymedium that is capable of storing or encoding a set of instructions forexecution by the machine that cause the machine to perform any one ormore of the methods described herein. The term “computer-readablestorage medium” shall accordingly be taken to include, but not belimited to, solid-state memories, and optical and magnetic media.

Some portions of the detailed descriptions above are presented in termsof algorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “identifying,” “determining,”“storing,” “adjusting,” “causing,” “returning,” “comparing,” “creating,”“stopping,” “loading,” “copying,” “throwing,” “replacing,” “performing,”or the like, refer to the action and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (electronic) quantities within the computersystem's registers and memories into other data similarly represented asphysical quantities within the computer system memories or registers orother such information storage, transmission or display devices.

Examples of the present disclosure also relate to an apparatus forperforming the methods described herein. This apparatus may be speciallyconstructed for the required purposes, or it may be a general purposecomputer system selectively programmed by a computer program stored inthe computer system. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding optical disks, CD-ROMs, and magnetic-optical disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs,magnetic disk storage media, optical storage media, flash memorydevices, other type of machine-accessible storage media, or any type ofmedia suitable for storing electronic instructions, each coupled to acomputer system bus.

The methods and displays presented herein are not inherently related toany particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear as set forth in thedescription below. In addition, the scope of the present disclosure isnot limited to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the present disclosure.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementation exampleswill be apparent to those of skill in the art upon reading andunderstanding the above description. Although the present disclosuredescribes specific examples, it will be recognized that the systems andmethods of the present disclosure are not limited to the examplesdescribed herein, but may be practiced with modifications within thescope of the appended claims. Accordingly, the specification anddrawings are to be regarded in an illustrative sense rather than arestrictive sense. The scope of the present disclosure should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

What is claimed is:
 1. A method, comprising: receiving, by a processor,a request to create a snapshot of a virtual machine running on a hostcomputer system; responsive to detecting an attempt to modify a virtualmemory page associated with the virtual machine, copying the virtualmemory page to a queue residing in a random access memory (RAM) of thehost computer system; asynchronously with respect to copying the virtualmemory page to the queue, retrieving the virtual memory page from thequeue; wherein responsive to retrieving the virtual memory page from thequeue, compressing the virtual memory page; and writing the virtualmemory page to a disk of the host computer system.
 2. The method ofclaim 1, wherein writing the virtual memory page to the disk isperformed without interrupting a user session associated with thevirtual machine.
 3. The method of claim 1, wherein copying the virtualmemory page to the queue is performed by a first processing thread andwriting the virtual memory page to the disk is performed by a secondprocessing thread.
 4. The method of claim 1, further comprising:responsive to retrieving the virtual memory page from the queue,compressing the virtual memory page; copying the virtual memory page toa second queue residing in the RAM; and retrieving the virtual memorypage from the second queue.
 5. The method of claim 1, furthercomprising: responsive to retrieving the virtual memory page from thequeue, determining a value of a function of at least one of: availablehost memory or host processor utilization; and responsive to determiningthat the value meets a threshold, compressing the virtual memory page.6. The method of claim 1, further comprising: responsive to receivingthe request, modifying, in a paging table, a writable flag associatedwith the memory page.
 7. The method of claim 1, wherein writing thevirtual memory page to the disk is performed by a system call bypassingan input/output (I/O) cache of the host computer system.
 8. A system,comprising: a memory; and a processor coupled to the memory, theprocessor configured to: receive a request to create a snapshot of avirtual machine running on a host computer system; responsive todetecting an attempt to modify a virtual memory page associated with thevirtual machine, copy the virtual memory page to a first queue residingin a random access memory (RAM) of the host computer system; responsiveto retrieving the virtual memory page from the first queue, compress thevirtual memory page to produce a compressed memory page; copying thecompressed memory page to a second queue residing in the RAM; andresponsive to retrieving the compressed memory page from the secondqueue, write the compressed memory page to a disk of the host computersystem.
 9. The system of claim 8, wherein compressing the virtual memorypage to the disk is performed asynchronously with respect to copying thevirtual memory page to the first queue.
 10. The system of claim 8,wherein writing the virtual memory page to the disk is performedasynchronously with respect to copying the virtual memory page to thesecond queue.
 11. The system of claim 8, wherein copying the virtualmemory page to the first queue is performed by a first processingthread, compressing the virtual memory page is performed by a secondprocessing thread, and writing the virtual memory page to the disk isperformed by a third processing thread.
 12. The system of claim 8,wherein the processor is further configured to: responsive to receivingthe request, modifying, in a paging table, a writable flag associatedwith the memory page.
 13. The system of claim 8, wherein writing thevirtual memory page to the disk is performed by a system call bypassingan input/output (I/O) cache of the host computer system.
 14. Anon-transitory computer-readable storage medium comprising executableinstructions that, when executed by a processor, cause the processor to:receive a request to create a snapshot of a virtual machine running on ahost computer system; responsive to detecting an attempt to modify avirtual memory page associated with the virtual machine, copying thevirtual memory page to a queue residing in a random access memory (RAM)of the host computer system; wherein responsive to retrieving thevirtual memory page from the queue, compressing the virtual memory page;asynchronously with respect to copying the virtual memory page to thequeue, retrieving the virtual memory page from the queue; writing thevirtual memory page to a disk of the host computer system.
 15. Thenon-transitory computer-readable storage medium of claim 14, whereincopying the virtual memory page to the queue is performed by a firstprocessing thread and writing the virtual memory page to the disk isperformed by a second processing thread.
 16. The non-transitorycomputer-readable storage medium of claim 14, further comprisingexecutable instructions that, when executed by the processor, cause theprocessor to: responsive to retrieving the virtual memory page from thequeue, compress the virtual memory page.
 17. The non-transitorycomputer-readable storage medium of claim 14, further comprisingexecutable instructions that, when executed by the processor, cause theprocessor to: responsive to retrieving the virtual memory page from thequeue, compress the virtual memory page; copy the virtual memory page toa second queue residing in the RAM; and retrieve the virtual memory pagefrom the second queue.
 18. The non-transitory computer-readable storagemedium of claim 14, wherein writing the virtual memory page to the diskis performed by a system call bypassing an input/output (I/O) cache ofthe host computer system.
 19. The non-transitory computer-readablestorage medium of claim 14, further comprising executable instructionsthat, when executed by the processor, cause the processor to: responsiveto receiving the request, modify, in a paging table, a writable flagassociated with the memory page.